Tactile feedback interface device

ABSTRACT

A system includes a host computer configured to generate a graphical interface that includes a graphical object, obtain at least one spoken utterance, control an interaction of the graphical object within the graphical interface based on the spoken utterance, and generate an activating signal based on the spoken utterance. An interface device is configured to receive the spoken utterance, provide the spoken utterance to the host computer, and control the graphical object within the graphical interface based on the provided spoken utterance. The graphical object includes a graphical representation within the graphical interface. An actuator is disposed within a housing of the interface device, thereby protecting the actuator from contact by the user, and is configured to receive the activating signal from the host computer. The activating signal causes the actuator to impart a force via the housing.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 11/580,452, filed Oct. 13, 2006, which is a continuation ofU.S. patent application Ser. No. 10/460,157, filed Jun. 16, 2003 andissued as U.S. Pat. No. 7,755,602 on Jul. 13, 2010, which is acontinuation application of U.S. application Ser. No. 10/186,342, filedJun. 27, 2002 and now abandoned, which is a continuation of U.S. patentapplication Ser. No. 09/838,052, filed Apr. 18, 2001 and issued as U.S.Pat. No. 6,424,333 on Jul. 23, 2002, which is a continuation of U.S.patent application Ser. No. 09/561,782, filed on May 1, 2000 and issuedas U.S. Pat. No. 6,275,213 on Aug. 14, 2001, which is a continuation ofU.S. patent application Ser. No. 09/066,608, filed Apr. 24, 1998 andissued as U.S. Pat. No. 6,088,017 on Jul. 11, 2000, which is acontinuation of U.S. patent application Ser. No. 08/565,102, filed Nov.30, 1995, now abandoned, and all of which are incorporated herein byreference in their entireties.

FIELD

This invention relates to a man-machine interface and in particular toan interface that provides tactile sensation to a user.

BACKGROUND

Virtual reality (VR) is an immersive environment which is created by acomputer and with which users have real-time, multisensorialinteractions. Typically, these interactions involve some or all of thehuman senses through either visual feedback, sound, force and tactilefeedback (i.e. reflection), smell and even taste. The key to immersiverealism is the capacity of the user to use his/her hand to interactivelymanipulate virtual objects. Unfortunately, the majority of existingcommercial virtual reality systems use hand-sensing devices that provideno haptic feedback. Nevertheless, some efforts have been made to providemeans for presenting force and tactile information to the user's hand.By force information, it is meant the application of a set force to aselected part of the hand, for example, a finger. By tactileinformation, it is meant the application of a stimuli, e.g., avibration, to a selected part of the hand, e.g., a fingertip pad. Thisstimulus, could simulate surface texture or dynamic conditions at thecontact, for example. A few examples of existing force reflectingdevices are the EXOS SAFiRE™, the Master II Hand Master device atRutgers university, the PERCRO Force-Reflecting Hand Master and theSarcos TOPS Force-Reflecting Hand Master. Some tactile feedback devicesthat have been developed include the PERCRO Position-Sensing and TactileFeedback Hand Master and the EXOS TouchMaster™.

Virtual reality is not the only field where it is desirable to feed backforce and tactile information to a human user/operator. Another commonarea is telerobotics. Some of the devices mentioned above are also oftenused as telerobotics interfaces. Some examples in the literature offeedback devices designed more specifically for telerobotics include thetactile shape sensing and display system developed by Kontarinis et al.,the voice-coil based tactile feedback device used by Patrick et al. andthe pin-based tactile display array developed by Kaczmarek andBach-y-rita. Other applications for a vibrotactile unit of the subjectinvention include, but are not limited to, gesture recognition, musicgeneration, entertainment and medical applications.

In an ideal case, it would be desirable to provide full force andtactile feedback to a user to make the virtual reality or teleroboticexperience as realistic as possible. Unfortunately, most force feedbackdevices are cumbersome, heavy, expensive and difficult to put on andremove. Many of the tactile feedback solutions are also cumbersome,complex and fragile. Additionally, some of the tactile feedback devicesdescribed in the literature, such as small voice coils mounted todirectly contact the skin, tend to numb the skin after only a fewseconds of operation and then become ineffective as feedback devices.

SUMMARY

An object of the invention is a man-machine interface which may beemployed in such areas as interactive computer applications,telerobotics, gesture recognition, music generation, entertainment,medical applications and the like. Another object of the invention is amass which is moved by a “mass-moving actuator” which generates avibration that a user can feel. Yet another object of the invention isthe generation of an activating signal to produce the vibrations eitheras a result of the user's state or as a result of environmentalconditions, whether virtual or physical. Still another object of theinvention is vibrating the bone structure of a sensing body part, aswell as skin mechanoreceptors, to provide feedback. Yet still anotherobject of the invention is the complex actuation of vibratory devices.

The tactile sensation that a user feels is generated by a vibrotactileunit mounted on, or in functional relation to, a sensing body part of auser by a fastening means. In one embodiment, the vibrotactile devicecomprises a mass connected eccentrically to a mass-moving actuator shaft(i.e. the center of mass of the mass is offset from the axis ofrotation). Energizing the mass-moving actuator causes the shaft to turn,which rotates the eccentric mass. This rotating mass causes acorresponding rotating force vector. A rapidly rotating force vectorfeels to the user as a vibration. A slowly rotating force vector feelslike a series of individual impulses. For a small number of rapidrotations, the rotating force vector feels like a single impulse. Wewill use the term “vibration” to denote a change in force vector (i.e.,direction or magnitude). Examples of vibrations include, but are notlimited to a single impulse, a sinusoidal force magnitude, and otherfunctions of the force vector. We use the term “tactile sensation” torefer to the feeling perceived by a user when their sensing body partexperiences vibrations induced by a vibrotactile unit.

A signal processor interprets a state signal and produces an activatingsignal to drive the mass-moving actuator. The variable components of thestate signal may be physical (e.g., measured), or virtual (e.g.simulated, or internally generated); they may vary with time (e.g., thestate variables may represent processes); and they may be integer-valued(e.g., binary or discrete) or real-valued (e.g., continuous). The signalprocessor may or may not comprise a computer which interprets andfurther processes the state signal. The signal processor comprises asignal driver which produces an activating signal supplying power to, orcontrolling the power drawn by, the vibrotactile unit. The power may be,but is not restricted to, electric, pneumatic, hydraulic, and combustivetypes. The driver may be, but is not restricted to, an electric motorcontroller comprising a current amp and sensor for closed loop control,a flow valve controlling the amount of a pressurized fluid or gas, aflow valve controlling the amount of fuel to a combustion engine and thelike. The details of such a signal processor and mass-moving actuatorare common knowledge to someone skilled in the art.

The state signal may be generated in response to a variety ofconditions. In one embodiment, one or more sensors measuring physicalconditions of the user and/or the user's environment may generate one ormore components of a physical state signal. In another embodiment, acomputer simulation may determine the one or more components of avirtual state signal from a simulated (e.g., virtual) state orcondition. The virtual state may optionally be influenced by a physicalstate. The virtual state includes anything that a computer or timingsystem can generate including, but not restricted to, a fixed time froma previous event; the position, velocity, acceleration (or other dynamicquantity) of one or more virtual objects in a simulation; the collisionof two virtual objects in a simulation; the start or finishing of acomputer job or process; the setting of a flag by another process orsimulation; combinations of situations; and the like. The virtual statesignal is a machine-readable measurement of the virtual state variables.

The physical state signal is measured from physical state variables.These variables have relevance to the physical state of a body part ofthe user or the user's physical environment. The physical statevariables includes any measurable parameter in the environment or anymeasurable parameter relating to a body part of the user. Some examplesof measurable physical parameters in an environment include but are notrestricted to, the state of a body part, the position of objects in theenvironment, the amount of energy imparted to an object in theenvironment, the existence of an object or objects in the environment,the chemical state of an object, the temperature in the environment, andthe like. The state of a body part may include the physical position,velocity, or acceleration of the body part relative to another body partor relative to a point in the environment. The state of a body part mayalso include any bodily function, where the measured state signal mayinclude the output from an electroencephalograph (EEG),electrocardiograph (ECG), electromyograph (EMG), electrooptigraph (EOG)or eye-gaze sensor, and sensors which measure joint angle, heart rate,dermal or subdermal temperature, blood pressure, blood oxygen content(or any measurable blood chemical), digestive action, stress level,voice activation or voice recognition, and the like. The user's voicemay constitute a measured physical state variable, where his spokenwords are sensed and/or recognized to generate a correspondingactivating signal. The physical state signal is a machine-readablemeasurement of the physical state variables.

The state signal is presented to the signal processor which interpretsthe state, and then determines how and when to activate the vibrotactileunits accordingly. The signal processor produces an activating signalwhich may be in response to an event it interprets from the statesignal. Examples of events include contact, gestures, spoken words,onset of panic or unconsciousness, and the like. The interpretation ofthe state signal may or may not be a binary event, i.e. the simplechanging of state between two values. An example of a binary event iscontact vs. non-contact between two virtual or real objects. The processof interpreting may include any general function of state variablecomponents. The interpretation function may produce an output controlvalue which is integer or real-valued. A non-binary-valuedinterpretation output typically relates to the signal processorproducing a non-binary activation signal.

By varying the functional form of the activation signal, the type offeedback that the vibrotactile device generates may also be varied. Thedevice may generate a complex tactile sensation, which is defined to bea non-binary signal from a single or multiple vibrotactile units.Examples of complex tactile sensations include (1) varying the amplitudeof vibration with a profile which is non-uniform over time; (2) varyingthe frequency of vibration; (3) varying the duration of impulses; (4)varying the combination of amplitude and frequency; (5) vibrating two ormore vibrotactile units with a uniform or non-uniform amplitude profile;(6) sequencing multiple vibrotactile units with different amplitude orfrequency profiles; and the like.

The frequency and amplitude of the vibration or impulse may be changedby modifying the activating signal to the mass-moving actuator. Thefrequency and amplitude may also be controlled by increasing the mass orby changing the radius of gyration (e.g. changing its eccentricity). Forexample, the mass may be changed by pumping fluid into an eccentricallyrotating container. The sense of frequency that the user perceives maybe changed independently of the amplitude by modulating the power to thevibrotactile unit at a variable frequency. This technique is calledamplitude modulation, which is common knowledge to those skilled in theart. This change in frequency and amplitude may be used to conveycomplex, compound or other forms of information to the user.

Sensors may be mounted on the vibrotactile unit or the sensing body partto determine the frequency and amplitude of vibration sensed by theuser. A feedback control loop may be added which uses this informationto more tightly control the frequency and amplitude, or to reach peakefficiency at the resonant frequency of the collective vibratingdevice-body system.

Examples of a sensing body part on which the vibrotactile unit may bemounted, or in functional relation to the vibrotactile unit, include,but are not limited to: the distal part of a digit, the dorsal (back)side of a phalanx or metacarpus, palm, forearm, humerus, underarm,shoulder, back, chest, nipples, abdomen, head, nose, chin, groin,genitals, thigh, calf, shin, foot, toes, and the like. A plurality ofvibrotactile units may be disposed on or near different sensing bodyparts, and may be activated in unison or independently.

Each vibrotactile unit may be affixed to the body by a fastening means.The fastening means is defined to be the means of attaching thevibrotactile unit to a sensing body part, transmitting (and possiblymodifying) the vibrations created by the vibrotactile unit. This meansmay be one that is flexible such as a strap made of cloth or softpolymer, or rigid, such as metal or hard polymer which grabs or pinchesthe flesh, skin or hair. The fastening means may also include gluing ortaping to the skin or hair, or tying with a string or rope around alimb, or attaching to clothes with VELCRO®. or similarly functionalmeans. A vibrotactile unit may also be attached to another structurewhich is then attached to the body part with the same means justmentioned. The vibrations generated by the actuator may be transmittedto the sensing body part by the structure (rigid or non-rigid), orthrough a linkage transmission or a fluid transmission.

The eccentric mass need not be mounted directly onto a motor shaft. Amechanical transmission may rotate the mass on a different shaft thanthe motor shaft. The mass-moving actuator rotates this shaft. Fluidssuch as air and liquids may also transmit the motion from a power sourceto the rotating eccentric mass. Changing magnetic fields may also beemployed to induce vibration of a ferrous mass.

As previously mentioned, state signals may relate to a physical orvirtual state. When the state represents a physical condition, thesubject invention includes a state measurement sensor which produces astate signal. This state measurement sensor may measure some property ofthe sensing body part. Recall that the body part associated withreceiving the vibrotactile stimulation is called the sensing body part,the body part associated with producing the activating signal is calledthe measured body part. The signal processor may receive signals fromthis sensor such as a tactile, position, bend, velocity, acceleration ortemperature sensor and generate an activating signal. In this way, theuser may receive feedback based on his actions or physical state. Forexample, the vibrotactile device may be used to train the user to dosome physical motion task. In this case, the position or motion of thebody part which is to do the motion task is measured by the statemeasurement sensor and is also the sensing body part. Direct stimulationto the body part being trained enhances the training of the task.Complex actuation in the form of a function of different levels offrequency or amplitude may inform the user whether his actions arecorrect or incorrect; the level of correctness may correspond to thelevel of frequency or amplitude.

In addition, the sensing body part (which is also the measured bodypart) may have a graphical representation shown to the user. The usermay also be presented with visual, auditory, taste, smell, force and/ortemperature cues to his actions in combination with the vibrotactilecues provided by the subject invention. The user may be immersed in avirtual environment. The user may see a graphical representation ofhis/her body part interact with virtual objects and simultaneously feela corresponding tactile sensation simulating the interaction. Forexample a user may have his/her fingers be the sensing and measured bodyparts. The user may then see his/her virtual hand in the virtualenvironment contact a virtual object. The user would then feel anincreasing vibratory stimulation on his/her physical fingertip as heincreased the virtual pressure on the virtual object using the virtualfingertip.

As previously discussed, using the vibrotactile device of the subjectinvention, a user may receive tactile sensations based on the state ofhis body parts. In the previous case the state included the position,and other dynamic quantities, of the body parts. In certainapplications, the measured body part is the same as the sensing bodypart (the list of possible sensing body parts mentioned earlier alsoapplies to measured body parts); in other applications they aredifferent body parts. When the measured body part is different than thesensing body part, the subject invention acts as a coupling device whichrelates the sensing body part and the measured body part.

In another application, the user may receive tactile feedback as aresult of the conditions of a computer simulated environment, notnecessarily related to the user's actions or state. The vibrotactileunits with varying actuation levels may be used to simulate a variety ofcontact situations, e.g., contact with fluids and solids, and contactswhich are momentary or continuous. For example, a user immersed in acomputer simulated virtual environment may feel simulated fluid (likeair or water) across his body. In such a simulation, an array ofvibrotactile units may vibrate in sequence to correspond to a pressurewave hitting the corresponding parts of the body; the amplitude of thevibration may vary to correspond to different levels of pressure beingsimulated. A user may also feel a virtual object that comes into contactwith a portion of his virtual body. The user may feel a virtual bugcrawl up his virtual arm by sequencing an array of vibrotactile units.To accompany the tactile sensations received by the user which areuncorrelated with his actions, the user may be presented with visual,auditory, taste, smell, force, temperature and other forms of feedbackin order to enhance the realism of the simulated environment.

In yet another application of the vibrotactile device, a group of usersmay receive tactile sensations. In one example, users may wearindividual vibrotactile units, or they may also share vibrotactile unitsas follows. A tactile sensation may be shared by one or more usersmaking physical contact with the sensing body part of another user. Forexample, one user may wear vibrotactile units on the backs of hisfingers. A second user, not wearing any vibrotactile units, may obtainvibrotactile feedback transmitted via the first user when the first userplaces the palmar side of his fingers on a sensing body part of thesecond user. The activating signal for each vibrotactile unit may becomputer controlled via either user's actions or through a computersimulated event. In a second example, a group of users may each receiveidentical tactile feedback through individually mounted vibrotactileunits. The common activating signal may correspond to measured bodyparts from a single, optionally separate, user. Different users may alsobe responsible for producing the common activating signal for one ormore vibrotactile units. For instance, the movement of one user's armmay control the vibrotactile unit on each user's arm; and the voice of asecond user may control the vibrotactile unit on each user's back; theeye-gaze of three other users may control the vibrotactile unit uponwhich they stare in unison. An example application of a single usercontrolling many user's vibrotactile sensations is a new form ofentertainment where a performer creates vibrotactile sensations for anaudience.

In a preferred embodiment, the vibrotactile units are affixed to aninstrumented glove, such as the CyberGlove™ manufactured by VirtualTechnologies of Palo Alto, Calif, USA. The CyberGlove has sensors in itwhich measure the angles of the joints of the hand. The fingertips ofthe CyberGlove are open so that the user may reliably handle physicalobjects while wearing the glove. The open fingertips allow the user tofeel the sensations of real objects in conjunction with the generatedvibrotactile sensations. The fingertips need not be open, they may befully enclosed as in the 22-sensor model of the CyberGlove. Themass-moving actuator of each vibrotactile unit is encased in acylindrical housing and mounted onto the glove on each of the fingersand thumb, and on the palmar side of the hand. Each mass-moving actuatoris composed of a small DC motor with an eccentric mass mounted rigidlyonto the shaft of the motor. The casing is made of tubular plastic andserves to protect the motion of the mass from the user and protect theuser from the rotating mass. The casing may be made of any rigid orsemi-rigid material including but not limited to steel, aluminum, brass,copper, plastic, rubber, wood, composite, fiberglass, glass, cardboard,and the like. The casing may form a solid barrier, a wire-mesh, grid orcolumn-like support capable of transmitting vibrations from themass-moving actuator to the fastening means. The instrumented gloveinforms a computer of the position of the user's hand and fingers. Thecomputer, which is part of the signal processor, then interprets thishand state signal (and any virtual state signal if the application callsfor it). The computer then generates a control signal, which whenprocessed by the driver, activates the actuators to create tactilesensations.

One feature of the embodiment of the subject invention just described,which employs an eccentric mass, is that the energy imparted into thesystem can be less than the energy required when using electromagneticcoils (such as the speaker voice coils used by Patrick et al. and theEXOS TouchMaster). Energy is stored as rotational inertia in theeccentric mass, whereas the voice-coil-based systems lose all inertialenergy each time the coil change directions.

Another feature of the subject invention is that vibrating the bonestructure of a body part, as well as skin mechanoreceptors, has anadvantage over stimulating just the skin mechanoreceptors (such asMeissner, Merkel, Ruffini and Pacinian corpuscles) in that the nerves donot get easily overstimulated and do not become numb. In addition, theform of information to the user is closer to a physical contactsensation where the muscles and joints are stimulated, as is done byfull force feedback systems. As a result, the vibrotactile units neednot be attached to a body part which has sensitive skinmechanoreceptors. For example, a vibrotactile unit may be attached to afingernail or an elbow.

In an embodiment in which a user is immersed in a computer simulatedenvironment, actuation of vibrotactile units can approximate thesensation of touching physical objects as full force feedback devicesdo. The deep impulsive sensation in the muscles and joints generated bythe vibrotactile units simulates the change in proprioceptive state asthe user touches a virtual object. The subject invention providesnumerous advantages over a sustained force feedback device. For example,because of its simplicity, the vibrotactile device of the subjectinvention can be made smaller, lighter, less encumbering, more robustlyand cheaper.

The subject invention may be used in combination with a sustained forcefeedback device as provided by Kramer in U.S. Pat. No. 5,184,319, Kramerin U.S. patent application Ser. No. 08/373,531 (now U.S. Pat. No.5,631,861), Zarudiansky in U.S. Pat. No. 4,302,138, Burdea in U.S. Pat.No. 5,354,162, and Jacobus in U.S. Pat. No. 5,389,865. These patents andpatent applications are incorporated herein by reference. Such acombination can give a higher frequency response than that capable ofbeing generated by the sustained force feedback device and/or to reducethe cost and/or size of the full system. The subject invention may alsobe used in combination with other tactile feedback devices such asheating or cooling devices, bladder devices or voice coils.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1a is a perspective view of an electric mass-moving actuator withan eccentric mass attached to its shaft.

FIG. 1b is a perspective view of a mass-moving linear actuator with amass attached to its shaft.

FIGS. 2a and 2b are a cross-sectional side view and a perspective viewrespectively of an example of a vibrotactile unit.

FIG. 3 is a perspective view of the vibrotactile unit shown in FIG. 2bwhere the vibrotactile unit is attached to the palmar side of thefingertip.

FIG. 4 is another perspective view of a vibrotactile unit attached tothe dorsal side of the fingertip, where it makes contact with the nail.

FIG. 5 is a perspective view of a vibrotactile unit attached to thedorsal side of the proximal phalanx.

FIG. 6 is a perspective view of a vibrotactile unit attached to the palmof the hand.

FIG. 7 is a perspective view of a vibrotactile unit attached to thedorsal side of the metacarpus (the back of the hand).

FIG. 8 is a perspective view of a vibrotactile unit attached to the topof the foot.

FIG. 9 is a side view of a multitude of vibrotactile units attached to avariety of places on the head.

FIGS. 10a and 10b are front and back views respectively, of a multitudeof vibrotactile units attached to a variety of places on the body.

FIGS. 11a and 11b are perspective and front views respectively of afastening means where the palmar side of the fingertip receives greaterstimulation without the vibrotactile unit getting in the way ofmanipulation.

FIGS. 12a and 12b are perspective and front views respectively of afastening means where the palmar side of the fingertip receives greaterstimulation without the vibrotactile unit getting in the way ofmanipulation.

FIG. 13 is a perspective view of a fastening means where no casing isrequired for the vibrotactile unit because the moving mass is mountedaway from the finger thus reducing the possibility of interference.

FIG. 14 is a side view of an another fastening means where no casing isrequired for the vibrotactile unit because the moving mass is mountedaway from the finger thus reducing the possibility of interference.

FIG. 15 is a perspective view of a fastening means where thevibrotactile unit is attached to the body via a spring that is used toalter the amplitude and frequency of the sensed oscillations.

FIGS. 16a, 16b and 16c are front schematic views and 16 d is aperspective view of a vibrotactile unit where the radius of the gyrationof the eccentric mass increases as the angular velocity of the shaftincreases. FIGS. 16a, 16b, and 16c illustrate the principle, wherew₂>w1>0.

FIG. 17 is a schematic electrical-mechanical signal propagation diagram.

FIGS. 18a and 18b show an instrumented glove, in this case the VirtualTechnologies CyberGlove™, with both position sensors and vibrotactileunits.

FIGS. 19a and 19b show schematically two applications using a sensingglove with vibrotactile units attached to the fingers and back of thehand. FIG. 19a shows a virtual reality application. FIG. 19b shows atelerobotic application.

FIG. 20 illustrates schematically an example of a virtual environmentwith separate sensing and measured body parts.

FIGS. 21a and 21b are perspective drawings showing two applications ofthe invention for gesture recognition. In FIG. 21a , the gesture isstatic and in FIG. 21b , the gesture is dynamic (the index finger ismoving in a pre-determined fashion).

FIG. 22 is a schematic drawing showing a musical application.

FIG. 23 is a schematic drawing showing an entertainment or relaxationapplication.

FIGS. 24a and 24b is a schematic drawing showing a medical application.

FIGS. 25a, 25b and 25c are a schematic drawings illustrating anamplitude decoupling method.

FIG. 26a is a perspective drawing and FIG. 26b is a side view of avibrotactile unit with a controllable eccentricity. FIG. 26c is analternative transmission method that can be used to control theeccentricity of the mass.

FIG. 27 is an exemplary diagrammatic view of an application comprisingthe vibrotactile device, a virtual simulation and a physical simulation.

DETAILED DESCRIPTION

FIG. 1a shows one embodiment of a vibrotactile unit obtained byattaching an eccentric mass (101) to the shaft (102) of a small d.c.electric motor (100) which serves as the mass-moving actuator. Here themass is pie-shaped, however any other shape which offsets the center ofgravity from the axis of rotation, and thus provides eccentricity, maybe used. This eccentricity causes the force vector to change directionsduring rotation and thus induces vibrations in the unit. The mass may bemade of any material like steel, aluminium, plastic or fluid encased ina container, to name a few.

FIG. 1b shows another embodiment of a vibrotactile unit obtained byattaching a mass (103) to the shaft (104) of a linear mass-movingactuator (105). Here the mass is disk-shaped, however any other shapemay be used. The linear actuator moves the mass back and forth and thusinduces vibrations in the unit by suddenly accelerating and deceleratingit. The mass may be made of any material like steel, aluminium, plasticor fluid encased in a container, to name a few.

FIG. 2a and FIG. 2b are cross-sectional and perspective drawingsrespectively of an example of a casing (200) in which a mass-movingactuator (202) and a mass (201) are contained. Again, the eccentric mass(201) is attached to the shaft (204) of the electric motor. The casinghas a hole for the motor leads (203) to escape. The casing protects themoving mass from being disturbed by the user. It also protects the userfrom being hit by the mass. The casing may be made from any rigidmaterial, or variety of materials, such as aluminum, steel, plastic,glass, glass fiber composite etc. It is typically desirable to have asmall light weight mass-moving actuator, mass, casing and fasteningmeans so that the device may be as unencumbering as possible. From hereon, this embodiment of the invention will serve as the samplevibrotactile unit used in many of the subsequent figures.

FIG. 3 illustrates one fastening means for attaching the vibrotactileunit to a finger. In this example, the vibrotactile unit (300) isattached directly to the palmar side of the fingertip using a fasteningmeans (301). The fastening means may be made of either flexiblematerial, such as cloth, fabric, tape, VELCRO®. or a soft polymer, or itmay be made of a rigid material such as a metal, hard polymer or wood,to name a few. The fastening means need not encircle the fingerentirely, it may grab the finger by clamping or pinching, or by stickingto it such as with glue or tape. Also, if the user is wearing a glove,the vibrotactile unit may also be sewn onto the glove or bonded to itand need not be affixed directly to the hand. This will also be the casein the following figures which illustrate various ways to position thevibrotactile unit on the human body.

FIG. 4 illustrates another way of mounting the vibrotactile unit (400)onto the finger using a fastening means (401). In this case the unit ispositioned directly above the fingernail on the dorsal side of thefingertip (the sensing body part) in order to provide a distinctivetactile sensation, or vibratory stimulus. The unit may vibrate the nail,the flesh underneath and the bone with sufficient amplitude that thesensation is felt throughout the finger, not just locally at the skin.

FIG. 5 illustrates another way of mounting the vibrotactile unit (500)onto the finger using a fastening means (501). In this case the unit ispositioned on the dorsal side of the proximal phalanx. Since the unitgives sensation throughout the entire finger, touching virtual objectswith the palmar side of the hand will still give sensations to that sideeven though it is mounted on the back. When used in conjunction withmanipulating physical objects with the palmar side, the vibrationalsensation on the palmar side is enhanced. These features are not limitedto the proximal phalanx. Mounting to the dorsal side of any phalanx orlimb will produce the same effect.

FIG. 6 illustrates another way of mounting the vibrotactile unit (600)onto the user using a fastening means (601). In this case the unit ispositioned in the palm of the user's hand. If a glove (instrumented ornot) is worn, then the unit may also be mounted inside or outside of theglove in a pocket-like cavity, and need not be explicitly affixed to thehand.

FIG. 7 illustrates another way of mounting the vibrotactile unit (700)onto the user using a fastening means (701). In this case the unit ispositioned on the dorsal side of the metacarpus, or the back of the handAgain, if a glove (instrumented or not) is worn, then the unit may alsobe mounted inside or outside of the glove in a pocket-like cavity, andneed not be explicitly affixed to the hand.

FIG. 8 illustrates another way of mounting the vibrotactile unit (800)onto the user using a fastening means (801). In this case the unit ispositioned on the top of the user's foot. If a sock-like garment(instrumented or not) is worn, then the unit may also be mounted insideor outside of the garment in a pocket-like cavity, and need not beexplicitly affixed to the foot.

FIG. 9 illustrates another way of mounting vibrotactile units (900) ontothe user. In this example, the units are positioned on the user's head.If a hat-like garment (instrumented or not) is worn, then the units mayalso be mounted inside or outside of the suit in pocket-like cavities,and need not be explicitly affixed to the body. Examples of locationsinclude, but are not limited to, the temples (900), the forehead (901),the top of the head (902) and the back of the head (903).

FIG. 10a illustrates another way of fastening vibrotactile units(1000-1012) onto the user. In these examples the units are positionedall over the front and the side of the user's body. If a body suit(instrumented or not) is worn, then the units may also be mounted insideor outside of the suit in pocket-like cavities, and need not beexplicitly affixed to the body. By actuating a combination of actuators,the perception of the localization of the tactile sensation may becontrolled. For example, if the actuators on the forearm (1007) and onthe humerus (1005) actuate with equal intensity, the user may have theperception that there is a single source of sensation originatingin-between the two. This may apply to any combination of vibrotactileunits located anywhere on the body. This effect is also apparent whenmultiple vibrotactile units are activated in sequence. There is aperception that a single vibration has “moved” between the activatingvibrotactile units. The vibrotactile units displayed in the figure showexamples of a variety of candidate positions for attaching the units.Some of these positions include, but are not limited to, the forehead(1000), the shoulders (1001), the side of the arm (1003), the humerus(1005), the chest (1002), the nipples (1004), the abdomen (1006), theforearm (1007), the groin (1008), the hips (1009), the thighs (1010),the knees (1011) and the shins (1012).

FIG. 10b illustrates another way of fastening vibrotactile units(1020-1028) onto the user. In these examples the vibrotactile units arepositioned all over the back of the user's body. If a body suit(instrumented or not) is worn, then the vibrotactile units may also bemounted inside or outside the body suit in pocket-like cavities, andneed not be explicitly affixed to the body. The vibrotactile unitsdisplayed in the figure show examples of a variety of candidatepositions for attaching the units to the body. Some of these positionsinclude, but are not limited to, the back of the head (1020), the baseof the neck (1021), between the shoulder blades (1022), the back of thehumerus (1023), the back of the forearm (1025), the lower back (1024),the buttocks (1026), the back of the thighs (1027) and the calves(1028). The vibrotactile units in FIG. 10b may be combined with those inFIG. 10a as well. This plurality of vibrotactile units shows one waythat complex tactile sensations may be generated with multiplevibrotactile units.

FIGS. 11a and 11b show a vibrotactile unit mounted in such a way thatthe fingertip may be stimulated without the unit getting in the way ofmanipulation with the fingers. FIG. 11a shows a perspective view of theinvention and FIG. 11b shows a frontal view. A structure (1102), whichmay be opened or closed at the end, surrounds the fingertip. Thefastening means is comprised of three parts: part one affixing thefinger to the structure; part two affixing the vibrotactile unit to thestructure; part three is the structure (1102). Part one of the fasteningmeans (1103), which can be a flexible or rigid membrane, holds thefinger against the structure on the palmar side of the fingertip. Thispart can be adjustable, fixed, flexible or stretchable. In part two, thevibrotactile unit (1100) is mounted atop the structure, away from thepalmar side of the fingertip, using a means (1101) which can be aflexible or rigid membrane. In this manner, the vibrations from thevibrotactile unit may be transmitted through the structure directly tothe palmar side of the finger to provide a greater stimulation of thenerves local to the palmar side. In another embodiment, the structure(1102) and the vibrotactile unit casing (1100) can be made of one part,thus eliminating the need for a part two of the fastening means (1101).

FIGS. 12a and 12b show the vibrotactile unit mounted such that thefingertip can be stimulated without the unit getting in the way ofmanipulation with the fingers. FIG. 12a shows a perspective view of theinvention and FIG. 12b shows a side view. A structure (1202) is attachedto the palmar side of the fingertip. The fastening means is comprised ofthree parts: part one, affixing the finger to the structure; part two,affixing the vibrotactile unit to the structure; part three which is thestructure (1202). Part one (1203) may be a flexible or rigid membrane.This part may be adjustable, fixed, flexible or stretchable. In parttwo, the vibrotactile unit (1200) is mounted atop the structure, awayfrom the palmar side of the fingertip, using a means (1201) which can bea flexible or rigid membrane. In another embodiment, the structure(1202) and the vibrotactile unit casing (1200) can be made of one part,thus eliminating the need for part two of the fastening means (1201).

FIG. 13 shows a vibrotactile unit and fastening means where no casing isrequired for the mass-moving actuator/mass assembly. A small rigid orsemi-rigid structure (1302) elevates the vibrotactile unit above thefingertip in such a way that the finger cannot interfere with therotation of the eccentric mass (1301) about the main axis of the shaft(1304) of the mass-moving motor (1300). The structure (1302) is attachedto the fingertip using a strap (1303) which can be rigid or flexible andwhich can be either an integral or separate part of the structure.

FIG. 14 shows another vibrotactile unit and fastening means where nocasing is required for the mass-moving actuator/mass assembly. A smallrigid or semi-rigid structure (1404) elevates the vibrotactile unitabove the middle phalanx in such a way that the finger cannot interferewith the rotation of the eccentric mass (1402) about the main axis ofthe shaft (1403) of the mass-moving actuator (1401). The structure(1404) is attached to the middle phalanx using a strap (1405) which canbe rigid or flexible and which can be either an integral or separatepart of the structure.

FIG. 15 shows yet another vibrotactile unit (1500) and fastening meanssuch that the vibrotactile unit is connected to the fingertip via a formof spring (1501) in order to alter the amplitude and frequency of theperceived vibrations. A strap (1502) which can be a flexible or rigidmembrane holds the spring against the fingertip. The spring changes thenatural frequency of the vibrotactile unit. Alternatively, thevibrotactile unit/spring apparatus could be attached below the fingerinstead of above it. The spring may also be replaced by some form ofactuator to again control the amplitude and frequency and/or to extendthe range of amplitude and frequency. In addition a damper may beintroduced in combination with the spring or actuation system to furthercontrol and/or extend the amplitude and frequency of the perceivedvibrations. An electro-rheological fluid may be used in the damper tocontrol the damping term in the mechanical system.

FIGS. 16a, 16b, 16c, and 16d illustrate a modification to the way theeccentric mass is mounted to a shaft. The radius of the gyration K ofthe eccentric mass increases as the angular velocity of the shaftincreases. The top three drawings (FIGS. 16a, 16b, 16c ) illustrate theprinciple, where w₂>w₁>0, and the bottom perspective drawing (FIG. 16d )provides an implementation. In FIG. 16d , a structure (1601) is attachedthe shaft (1602) of the mass-moving actuator (1600). The structurecomprises a spring (1604) and mass (1603) assembly. At one end, thespring is attached to the inside of the structure and at the other endit is attached to the mass. The mass is free to move towards and awayfrom the shaft inside a guide in the structure. The radius of gyration Kis the distance between the center of gravity of the mass (1603) and themain axis of the mass-moving actuator shaft (1602). As the angularvelocity of the shaft increases, the centrifugal forces felt by the massincrease, causing it to stretch the spring further and increase theradius of gyration. This apparatus minimizes the angular inertia of thedevice at start-up and then gradually increases the eccentricity of themass so that larger vibrations can be obtained at higher angularvelocities. This relieves the stress on the bearings that hold the shaftand reduces the larger initial torque required to initiate rotation (asopposed to the torque required to maintain rotation). Alternatively, thepassive spring may be replaced by an active device which controls orsets the radius of gyration of the mass. The active device may comprisea shape memory alloy actuator or any other mechanism capable ofcontrolling the position of the mass.

FIG. 17 shows how the electrical and mechanical signals propagatethrough the tactile feedback control system in a preferred embodiment ofthe invention. The embodiment shown employs a d.c. servo motor (1701) asthe mass-moving actuator of a vibrotactile unit. A computer (1707), orother signal processing means, sends a digital value representing thedesired actuation level control signal to the digital-to-analog convert,D/A (1703). The analog output of the D/A is then amplified by a variablegain amplifier (1704) to produce an analog voltage activation signal.This voltage is placed across the servo motor, driving the motor at adesired angular velocity. The voltage signal may alternately beconverted to a current activation signal for driving the motor at adesired torque. Velocity damping of the servo control loop may beperformed by tachometer feedback (not shown). The computer (1707),digital-to-analog converter (1703), analog-to-digital converter, A/D(1702), bus (1709) and variable gain amplifier (1704) may be elements ofa signal processor. Digitized values from A/D (1702) from analog jointangle sensors (1705) provide the position information of the fingers(measured body parts) to the computer as a physical state signal. In avirtual environment application, the physical state signal may causemotion in a corresponding virtual hand. If one of the digits of thevirtual hand is found to be intersecting a virtual object, the computercalculates the virtual force to be applied to the virtual digit usingknowledge of the virtual object's shape and compliance. The computerthen causes an activation signal to be sent to the vibrotactile unitsmounted on the user's fingers (sensing body part) to convey tactileinformation about that virtual force. Strain gage, fiber optic,potentiometric, or other angle sensors may be used as analog joint anglesensors (1705). Strain gage angle sensors are disclosed in the Kramer etal. U.S. Pat. Nos. 5,047,952 and 5,280,265, which patents areincorporated herein by reference.

FIG. 18a and FIG. 18b illustrate a preferred embodiment of theinvention. An instrumented glove (1820) such as the CyberGlove™manufactured by Virtual Technologies of Palo Alto Calif., USA, hassensors (1807-1819) on it which measure the angles of the joints of thehand (the measured body parts). In the figures, the fingertips of theglove are open so that the user may handle physical objects while usingthe glove. This allows the user to feel the tactile sensations of realobjects which may then be used in conjunction with tactile sensationsgenerated from the subject invention. The vibrotactile units (1801-1806)are encased in cylindrical housings and fastened to the glove on each ofthe fingers (1801-1804), the thumb (1805) and on the palmar (1806) sideof the hand. The vibrotactile units are composed of a d.c. motor (item202 in FIG. 2) with an eccentric mass (item 201 in FIG. 2) mounted ontoits shaft (item 204 in FIG. 2). The casing is made of tubular plasticand serves to protect the motion of the mass from the user and protectthe user from the rotating mass.

FIG. 19a shows a user wearing a sensing glove (1900) which can measurehand formations as well as the spatial placement of the hand. Thesensing glove has vibrotactile units (1901) fastened to the fingers andto the back of the hand. The user receives visual feedback through agraphical representation of his hand on the computer monitor (1908). Thecomputer (1904) receives the state signal (information about the spatialplacement of the user's hand) through the sensors mounted on the glovevia a glove sensor interface (1902). When the virtual graphical hand(1906) touches (1907) a virtual object (1905) on the monitor, thecomputer sends a control signal to the vibrotactile unit driver (1903)which then sends the activation signal to the vibrotactile units (1901).

In a similar setup, FIG. 19b shows the same glove (1900) and computerinterface remotely controlling a robotic arm (1911) instead of a graphicdisplay. The robot has contact sensors on its gripper (1912) that detectwhen the robot touches physical objects (1913). The user controls therobot arm through the sensors on the glove which produce positionreadings of the fingers (measured body parts) which are sent to theglove interface device (1902) and then outputted to the computer (1904)which in turn sends the appropriate commands to the robot. Robotposition and contact information (1910) is then fed back to the computeras the state signal. The computer interprets this signal and decideswhat kind of vibrational feedback should be sent to the vibrotactileunit (1901) (other vibrotactile units not shown) on the user via thevibrotactile unit driver (1903). Force or pressure sensors may bemounted on the gripper instead of contact sensors. The user thenreceives vibrational feedback of varying levels depending on the forceor pressure on the object. This allows a teleoperator to perform tasksmore efficiently and safely, especially in handling delicate objectsthat would break under certain grip forces. The user does notnecessarily need to control the robot or the objects of contact in orderto use the tactile feedback. The vibrotactile device may act simply toinform the user of contact with the object whether or not as a result ofthe user's actions.

FIG. 20 illustrates an embodiment in a virtual reality context where themeasured body part (2003) is the foot, however, the vibrotactile unit(2001) is mounted on the finger which acts as the sensing body part(2002). The foot has a graphical object (2004) associated with it in thecomputer simulation. In this case, the graphical object looks like afoot as well. Motions of the foot are sent to the computer (2008) viathe body sensor interface (2006) and are reflected on the computermonitor (2009). When the computer (2008) determines that the graphicalfoot (2004) contacts a virtual object (2010), the computer interpretsthis state signal and sends a control signal to the vibrotactile unitdriver (2007) to activate the vibrotactile unit (2001) on the finger.This may be due to the user moving his foot so that the graphical footcontacts (2005) the virtual object, or the virtual object moving intothe graphical foot independent of the user's actions. The user thenneeds to correlate the contact of the virtual object with the sensationat the fingertip. While this does not seem as natural as vibrating thefoot as it makes contact, this illustrates the sensing body partseparate from the measured body part. This is necessary if the measuringbody part cannot coincide with the sensing body part, for example if themeasured body part is the eye-ball or if the measured body part is onanother user.

FIG. 21a and FIG. 21b illustrate a glove (2101) which contains bothposition sensors and vibrotactile units (2100). The Virtual TechnologiesCyberGlove™ is an example of a glove with appropriate position sensors.The sensors measure the spacial placement of the hand and fingers. Acomputer uses gesture recognition software to determine if apre-specified hand formation or motion has been gesticulated. In FIG.21a , the vibrotactile unit signals the user that a particular staticpose has been detected. A different vibrotactile sensation can begenerated in response to a recognized moving hand or arm gesture thatincludes dynamic motions (FIG. 21b ). This may also be useful intraining the gesture recognition software for the gestures to berecognized. In training the software, a user must repeatedly make thesame gesture to obtain some sort of average position since humans cannotrepeat gestures exactly. With the vibrotactile feedback, the user may betrained to better repeat his gestures while at the same time trainingthe recognition software to recognize his gestures. Better repetition ofthe gestures reduces the statistical distribution of the sensor readingsfor a given hand gesture which in turn may improve the performance ofthe recognition system.

FIG. 22 illustrates vibrotactile units in a musical application. Theunits are attached to a body suit or mounted directly onto the user'sclothing. Different regions on the body (2200-2210), which containgroupings of vibrotactile units (2211), may correspond to differentmusical instruments in an orchestra and serve to enhance the musicalexperience. For example, music produced by a cello produces proportionalvibrations on the user's thigh (2204) through the vibrotactile unitslocated in that body region. Similarly, the drums induce vibrations inthe units that stimulate the chest area (2201) and so on. Sections ofthe body containing multiple vibrotactile units corresponding to oneinstrument type may have individual vibrotactile units corresponding toindividual instruments. For example, the cello section of the body isshown to have the first chair cello on the upper thigh, and the secondchair cello on the lower thigh. The user may either be a passivelistener “feeling” the instruments as well, or he may be an activeparticipant in creating the music, receiving the vibrotactile sensationsas feedback.

FIG. 23 illustrates an entertainment application. In this case an arrayof vibrotactile units simulates water flow or wind. In this illustrationa user is lying on a couch (2300) and is immersed in a virtual beachscene and sees the beach through a head-mounted display (2302). The userhears ocean sounds through head-mounted earphones (2301) and feelswarmth from the sun through heat lamps (2303). The user then feels windsimulated by the vibrotactile units as they are pulsed in sequencecreating “waves” of sensation. For example, the wind could flow fromhead to toes by alternatively pulsing the vibrotactile units startingwith the ones on the head (2304) and ending with the ones on the toes(2305). Similarly, water is felt as pulsed waves (although perhaps oflarger amplitude), as the user swims through the virtual water. In thisfashion, the user may be relaxed or entertained.

FIG. 24a and FIG. 24b illustrate a medical application where, forexample, a user has injured a knee. A vibrotactile unit (2401) is usedin conjunction with bend sensors (2400) mounted on the knee duringphysical therapy sessions as shown in FIG. 24a . The vibrotactile unitnotifies the user when the knee is exercised appropriately and alertsthe user if the knee is flexed further than a safe limit prescribed by adoctor and thus improve recovery as is illustrated in FIG. 24b .Furthermore, the vibrotactile units, in conjunction with other sensors,may be used in any biofeedback application.

FIGS. 25a, 25b and 25c illustrate an approach for decoupling theamplitude and frequency components of the vibrations generated by aneccentric mass-based vibrotactile unit. In this embodiment, thevibrotactile unit comprises a rotary electric motor (2501), a massmounted eccentrically (2500), a sensor (2503) mounted on the shaft(2502) to determine the angular position of the shaft and a closed-loopcontrol system. Any other control law may be used that achieves a singlerotation of the shaft. One example is shown in FIG. 25a . The verticalaxis of the graph represents a normalized current, the horizontal axisrepresents the rotation of the axis in radians. This corresponds to thefollowing non-linear control law:I=1, (δ≧θ>π)I=−1, (π≧θ>2π−δ)I=0, (−δ>θ>δ)

With the initial conditions set so that the velocity is zero androtational position of the mass, θ, (in radians) is equal to a smallvalue, δ, (FIG. 25b ) sending current, I, to the motor in this mannerwould cause the mass to accelerate for half of a full rotation, up toθ+δ (FIG. 25c ), and decelerate for the other half of the rotationcoming to a stop between the −δ and +δ position in the ideal case. Theactual position of the −δ and the +δ position may have to vary dependingon the bandwidth of the control loop and the friction and damping of thesystem. The magnitude of vibration or impulse is set by the amplitude ofthe current, the frequency is set by repeating the above control at thedesired frequency. A simple feedback control loop (PID for example)could ensure the initial conditions are correct before each pulse. Thedetails of this are common knowledge to those skilled in the art.

FIG. 26a is a perspective drawing and FIG. 26b is a side view of avibrotactile unit with controllable eccentricity. A structure, such as aslip-disk (2602), is mounted on the shaft (2601) and is free to slideback and forth along the shaft. The slip-disk is attached to a linkage(2605) which connects it to the eccentric mass (2604). A positioningdevice (2603) controls the position of the slip-disk on the shaft, whichin turn affects the position of the mass (via the linkage) and thus itseccentricity. FIG. 26c is an alternative transmission method that can beused to control the eccentricity of the mass. In FIG. 26c , thetransmission comprises element (2606), which may be a flexible membraneor fluid inside a hollow shaft (2601) that is connected to the eccentricmass (2604) at one end and the sliding slip-disk (2602) at the other.Again, controlling the position of the disk along the shaft using thepositioning device (2603) affects the eccentricity of the mass. Element2606 may also be a fluid and 2604 a hollow container. As the fluid isforced through the tube (2601) by the slip-disk (2602), or by some otherpressure generating means, the container (2604) is filled with thefluid, thus, increasing the effective radius of gyration of the centerof mass of the fluid. By increasing the radius of gyration, by whichevermeans, it is possible to independently control the amplitude andfrequency of vibration of the vibrotactile unit.

FIG. 27 provides an exemplary block diagram of the components andfunctional relationship between the components comprising thevibrotactile device when used in an application. Although somecomponents are shown interrelated with a unidirectional arrow, thearrows may be bidirectional to provide bidirectional flow ofinformation. Additional arrows may be added between blocks to providefor communication between components. An application may includepresenting vibrotactile sensation to a body part of a user, where thevibrotactile sensation is associated with a virtual environmentsimulation (virtual simulation) and/or the physical state of a body(physical simulation), which may be the user's body, or another person'sbody. One or more virtual simulations may co-exist with one or morephysical simulations, and may be combined with further types ofsimulations to yield a vibrotactile sensation.

The exemplary virtual simulation (2722) of FIG. 27 comprises a computer(2706) with computer monitor (2709). To produce the virtual simulation,the computer typically generates, processes, executes or runs a computersimulation (2705) which is typically in the form of a computer softwareprogram. In the exemplary virtual simulation, the computer produces agraphical display of a virtual measured body part on the monitor, wherethe body part is shown to be a fingertip (2707) on a virtual hand(2708). The virtual simulation may internally generate an internal statewith a variety of state variables which comprise a virtual state signal(2703) to provide to a signal processor (2700). Exemplary virtual statevariables include position, velocity, acceleration, mass, compliance,size, applying force or torque, composition, temperature, moisture,smell, taste and other dynamical, structural, physical, electricalmetabolical, moodal, cognitive, biological and chemical properties ofvarious portions internal to, and on the surface of, a virtual measuredbody part. States variables may also denote functions of statevariables, such as contact of various parts of the virtual hand.

FIG. 27 also provides an exemplary physical simulation (2723) comprisinga state sensor (2718) and a physical measured body part (2717). In thisexample, the physical measured body part is depicted as a fingertip(2717) on a physical hand (2716). The state sensor measures the physicalstate (2719) of the physical measured body part and produces a physicalstate signal (2720) to provide to the signal processor (2700). Thephysical state signal (2720) is optionally provided to the computer(2706) and/or computer simulation (2705) of a virtual simulation (2722).Exemplary physical state variables include position, velocity,acceleration, mass, compliance, size, applying force or torque,composition, temperature, moisture, smell, taste and other dynamical,structural, physical, electrical, metabolical, moodal, cognitive,biological and chemical properties of various portions internal to, andon the surface of, a physical measured body part. States variables mayalso denote functions of state variables, such as contact of variousparts of the physical hand.

As shown in FIG. 27, for both the virtual simulation and the physicalsimulation, the signal processor (2700) receives a state signal andproduces an activating signal (2704). For the virtual simulation, thestate signal is the virtual state signal (2703); for the physicalsimulation, the state signal is the physical state signal (2720). Whenthe application comprises both a virtual simulation and a physicalsimulation, both a virtual state signal and a physical state signal maybe presented to the signal processor.

The signal processor may employ a digital or analog computer (2701) tointerpret one or more input state signals (e.g., a virtual or physicalstate signal) and produce a control signal (2721) which becomes theinput signal to a driver (2702). Using the control signal, the driverproduces the activating signal (2704). When the computer is absent fromthe signal processor, the driver determines the activating signal fromthe state signal or signals. Whether or not the computer is present inthe signal processor, the activating signal is provided in a form foruse by the mass-moving actuator (2710). For example, the driver maycomprise a motion control module, an operational amplifier, atransistor, a fluidic valve, a pump, a governor, carburetor, and thelike. In the case where the mass-moving actuator is an electric motor,the activating signal may be a voltage or current with sufficientelectrical power or current drive to cause the motor to turn.

In FIG. 27, the mass-moving actuator (2710) is shown to be an electricmotor which turns a shaft (2712) and an eccentric mass (2711). As themass is rotated, the entire assembly of 2710, 2711 and 2712 vibrates,producing vibrations (2713). Such vibrations are transmitted to asensing body part of a user who perceives a tactile sensation. In thefigure, the sensing body part is exemplified as the fingertip (2715) ofa hand (2714).

An exemplary application which comprises a virtual simulation and aphysical simulation is summarized as follows:

A fingertip of a physical hand corresponds to both the sensing body part(2715) and the physical measured body part (2717). An instrumentedglove, optionally comprising a joint angle sensor and spatial positionsensor, produces a physical state signal (2720) corresponding to theposition of the physical fingertip. The fingertip position is providedvia a wire (e.g., electrical or optical), computer bus or otherelectrical or optical connecting means to a computer (2706) running acomputer simulation (2705) in software. The fingertip position is oftenprovided in the form of digital values, typically as joint angles and/orvalues corresponding to the six possible spatial degrees of freedom. Thefingertip position may also be in the form of an analog voltage.

The computer simulation uses the physical state signal (2720)corresponding to the physical fingertip position (2719) to simulate theposition and motion of a virtual hand (2708) and fingertip (2707). Thecomputer simulation displays the virtual hand on a computer monitor(2709), in addition to displaying a second virtual object whoseattributes may correspond to a second physical measured object, such asa block, ball, car interior, engineering part, or other object. Thecomputer monitor may be a desk-top monitor, head-mounted monitor,projection monitor, holographic monitor, or any other computer generateddisplay means. The second physical measured object may also be a bodypart of the user or a second user.

When the user moves his hand, producing a movement of the virtual hand,the computer simulation detects a level of virtual contact between thevirtual fingertip (2707) and the second virtual object. The computersimulation then produces a virtual state signal (2703) where one statevariable denotes the level (e.g., amount of force) of the virtualcontact of the virtual fingertip. The virtual state signal istransmitted to the signal processor (2721) via a wire (e.g., electricalor optical), computer bus or other electrical or optical connectingmeans. The signal processor may exist in the computer (2706), in whichcase the virtual state signal is typically transmitted via the computerbus. When the signal processor exists in a separate enclosure from thecomputer, the virtual state signal is typically transmitted via a wire.

The signal processor may convert the state variable level of virtualcontact into an activating signal voltage proportional to the level ofvirtual contact force using a digital-to-analog converter followed by anoperational amplifier. The voltage is presented to the mass-movingactuator (2710) typically by a wire (e.g., electrical or optical) orother electrical or optical connecting means. The mass-moving actuatormay be a variable speed electric motor which spins eccentric mass (2711)on its shaft (2712). The electric motor with eccentric mass are housedin a plastic housing or casing which is affixed to the dorsal portion ofthe instrumented glove surrounding the fingertip, which fingertipcorresponds to both the physical measured body part and the sensing bodypart. The affixing is typically done with straps, sewing, gluing and thelike. As the eccentric mass rotates, the electric motor, mass, casingand sensing body part all vibrate, typically at a common frequency.Ideally, there is little vibrational attenuation between the mass andthe sensing body part such that as the eccentric mass rotates, theelectric motor, mass, casing and sensing body part all vibrate with thesame amplitude.

In the application just described, the user may perceive acause-and-effect relationship between motion of his fingertip and thelevel of vibration he feels. Thus, the position, compliance, mass, shapeand other attributes of a virtual object may be detected by movement ofthe user's fingertip, or by movement of the virtual object, inducingvarious vibrotactile responses. The vibrotactile device of the subjectinvention promotes a sensation of immersion in a virtual environmentwhere the user is able to interact with virtual objects as if he wereinteracting with physical objects in a physical environment.

Any publication or patent described in the specification is herebyincluded by reference as if completely set forth in the specification.

While the invention has been described with reference to specificembodiments, the description is illustrative of the invention and is notto be construed as limiting the invention. Thus, various modificationsand amplifications may occur to those skilled in the art withoutdeparting from the true spirit and scope of the invention as defined bythe appended claims.

What is claimed is:
 1. A system, comprising: a computing devicecomprising a memory and a processor communicatively coupled to thememory, wherein the processor is configured to executeprocessor-executable instructions stored in the memory to: generate agraphical interface that includes a graphical object; obtain at leastone spoken utterance; control an interaction of the graphical objectwithin the graphical interface based on the at least one spokenutterance; and generate an activating signal based on the at least onespoken utterance; a user interface device configured to receive the atleast one spoken utterance, provide the at least one spoken utterance tothe computing device, and control the graphical object within thegraphical interface based on the provided at least one spoken utterance,wherein the graphical object comprises a graphical representation withinthe graphical interface; and a vibrotactile device comprising anactuator disposed within a housing of the user interface device, therebyprotecting the actuator from contact by the user, the actuatorconfigured to receive the activating signal, wherein the activatingsignal causes the actuator to impart a force via the housing, theactuator comprising a rotating mass actuator having a shaft and aneccentric mass coupled to the shaft, wherein the rotating mass actuatorproduces the force by the mass rotating about the shaft around an axisof rotation and wherein a center of mass of the eccentric mass is offsetfrom the axis of rotation thereby causing the force when the eccentricmass rotates about the shaft.
 2. The system of claim 1, furthercomprising: a second user interface device configured to be operated bya second user; and a second actuator disposed within a housing of thesecond interface device, thereby protecting the second actuator fromcontact by the second user, the second actuator configured to receivethe activating signal, wherein the activating signal causes the secondactuator to impart a force to the second user.
 3. The system of claim 1,wherein the actuator includes a moving portion that causes the force,wherein the moving portion is protected from contact by the user.
 4. Thesystem of claim 1, wherein the user interface device further comprises:a fastener configured to fasten the user interface device to the user.5. The system of claim 1, wherein the user interface device comprises ahandheld interface device.
 6. The system of claim 1, wherein theactivating signal causes the actuator to impart the force that varies byfrequency.
 7. The system of claim 1, wherein the graphical interfaceincludes a second graphical object and wherein the interaction isbetween the graphical object and the second graphical object.
 8. Thesystem of claim 7, wherein the imparted force varies based on a virtualpressure applied to the second graphical object.
 9. The system of claim1, wherein the actuator comprises a linear actuator.
 10. The system ofclaim 1, wherein the force comprises a varying amplitude over a durationof the force.
 11. The system of claim 1, wherein the activating signalcauses the actuator to output the force as a series of pulses.
 12. Thesystem of claim 1, wherein the force produces a tactile sensation. 13.The system of claim 1, wherein the processing device is configured togenerate the activating signal based further on one or more states ofthe graphical object, the one or more states comprising a position, avelocity, or an acceleration of the graphical object within thegraphical interface.
 14. The system of claim 1, wherein the activatingsignal causes the actuator to impart one or more of visual, auditory,taste, smell, or temperature cues to the user.
 15. The system of claim1, wherein the housing is mounted onto a sensing instrument, the sensinginstrument comprising one or more sensors that measure one or morephysical conditions of the user.
 16. A non-transitory computer-readablestorage medium storing processor-executable instructions, theinstructions when executed configuring the processor to perform amethod, the method comprising: displaying a graphical interface, thegraphical interface including a graphical object; receiving a signalrepresentative of at least one spoken utterance from a user interfacedevice configured to control the graphical object in the graphicalinterface, wherein the graphical object comprises a graphicalrepresentation of at least a portion of a body of a user; controlling aninteraction of the graphical object within the graphical interface basedon the signal representative of the at least one spoken utterance;generating an activating signal based on the signal representative ofthe at least one spoken utterance; and transmitting the activatingsignal to the user interface device based on an interaction related tothe graphical object in the graphical interface, wherein the activatingsignal causes an actuator disposed within a housing of the userinterface device to output a force via the housing, the actuatordisposed within the housing to protect the actuator from contact by theuser and wherein the actuator comprises a rotating mass actuator havinga shaft and an eccentric mass coupled to the shaft, wherein the mass isrotated about the shaft around an axis of rotation to produce the force,wherein a center of mass of the eccentric mass is offset from the axisof rotation thereby causing the force when the eccentric mass rotatesabout the shaft.
 17. The non-transitory computer readable storage mediumof claim 16, wherein the graphical interface includes a second graphicalobject and wherein the interaction comprises a collision between thegraphical object and the second graphical object in the graphicalinterface.
 18. The non-transitory computer readable storage medium ofclaim 16, wherein the force produces a tactile sensation.
 19. Thenon-transitory computer readable storage medium of claim 16, wherein theforce varies by amplitude over a duration of the force.
 20. Thenon-transitory computer readable storage medium of claim 16, wherein theactivating signal causes the actuator to output the force as a series ofpulses.
 21. An apparatus comprising: an input configured to receive atleast one spoken utterance to control a graphical object within agraphical interface, wherein the graphical object comprises a graphicalrepresentation of at least a portion of a body of a user; at least oneprocessor configured to: generate a signal based on the at least onespoken utterance; cause the signal to be transmitted to a host computer;and receive from the host computer an activating signal based on thegenerated signal; and a vibrotactile device comprising an actuatorconfigured to output a force based on the activating signal incoordination with the graphical interface, wherein the actuator isdisposed within the housing, thereby protecting the actuator fromcontact by the user, and wherein the actuator comprises a rotating massactuator having a shaft and an eccentric mass coupled to the shaft,wherein the mass is rotated about the shaft around an axis of rotationto produce the force, wherein a center of mass of the eccentric mass isoffset from the axis of rotation thereby causing the force when theeccentric mass rotates about the shaft.
 22. A computing device,comprising: one or more processors configured to: generate a graphicalinterface, wherein the graphical interface comprises at least onegraphical object; obtain at least one spoken utterance from an interfacedevice; control the at least one graphical object based on the at leastone spoken utterance from the interface device; generate an activatingsignal based on the at least one spoken utterance from the interfacedevice, wherein the activating signal is configured to cause an actuatorto impart a force; and provide the activating signal such that theactuator is caused to impart the force, the actuator comprising arotating mass actuator having a shaft and an eccentric mass coupled tothe shaft, wherein the mass is rotated about the shaft around an axis ofrotation to produce the force and wherein a center of mass of theeccentric mass is offset from the axis of rotation thereby causing theforce when the eccentric mass rotates about the shaft.
 23. The computingdevice of claim 22, wherein the at least one spoken utterance comprisesat least one spoken word.